Apparatus and methods relating to image coding

ABSTRACT

A product has a coding pattern on its surface. The coding pattern may include symbols representing at least two different values and where each symbol comprises a raster point and at least one marking. The raster point is included in a raster that extends over the surface and the value of each symbol is indicated by the placement of the marking in relation to the raster point. The markings may each have a spatial extent that, in combination, may at least partially form the image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority benefits based on Swedish PatentApplication No. 0000950-6, filed Mar. 21, 2000, and U.S. patentapplication Ser. No. 09/812,901, filed Mar. 21, 2001, the technicaldisclosures of each of which are hereby incorporated herein byreference.

FIELD OF THE INVENTION

The present invention relates to pattern recognition and detection.Specifically, the present invention relates to methods and apparatusesfor reading information from images stored as a pattern on a surface.

BACKGROUND OF THE INVENTION

In many contexts, both commercial and scientific, it is important tocreate an image on a surface, where the image contains as muchinformation as possible. In this context, images can comprise, forexample, photographically produced images of real objects andartificially generated images where information is represented bystructures of varying blackness. In addition to the information, whichis directly visible from the varying blackness over the image, it can beof use to add information to the image that is not directly apparentfrom the structures of varying blackness.

In U.S. Pat. No. 5,315,098 methods and means are illustrated whichencode digital data in the angular orientation of circularly asymmetrichalftone dot patterns that are written into the halftone cells ofdigital halftone images.

A drawback of the invention presented in U.S. Pat. No. 5,315,098 is thatits use is restricted to applications where the halftone cells are ofconstant spatial extension across an image. Another restriction is thatthe halftone information in the cells is restricted in spatial extent inthat it must have an elongated shape. This means that each halftone cellmust contain an empty area thus reducing the overall dynamic range ofthe image.

It is also of importance to be able to read and interpret thisinformation in the images applied to the surface, by means of readingapparatus. Examples can extend from applying and reading images on papersurfaces to applying and reading information from surfaces on productssuch as containers etc.

SUMMARY OF A FEW ASPECTS OF THE INVENTION

Systems, methods, and apparatus consistent with the present inventionmay employ coding patterns applied to “products” such as paper, computerdisks, maps, or like material, which can hold an image. The codingpattern in accordance with invention may be in the form of an image thatmay be produced with the aid of a screen-printing technique.

A product according to the invention thus may employ a surface with animage in the form of a coding pattern. The coding pattern may includesymbols representing at least two different values, wherein each symbolincludes a raster point and at least one marking, the raster point beingincluded in a raster extending over the surface. The value of eachsymbol may be indicated by the placement of the marking in relation to araster point and the markings may each have a spatial extent that, incombination, at least partially forms the image.

In slightly more specific terms, these two aspects of the invention canbe described by an original image that may be printed, for example, in anewspaper or other publication, normally being printed according to ascreen-printing technique which consists of the image being printed as aplurality of small points which are regularly arranged in a raster(i.e., a grid network with one point at each crossing between twolines). Different degrees of density i.e., graytones, may be produced inthe image by the points being made to be of different size. In an areathat is to be black, the points may be made large and in an area that isto be light, the points may be made smaller. One concept according tothe invention may be to code additional information, for example, in theform of positions, text, numbers etc., in the image by the center ofgravity of each point being displaced from its normal position in thegrid network. For example, if the point is displaced a given distanceupward from the normal position in the grid network, the point may codethe value 00, the same distance to the right the point codes the value10, the same distance downward may give the value 11 and the samedistance to the left may give the value 01. A set of displaced pointstogether may give binary numbers that code information. Since the pointsmay be relatively small and the displacements may also be relativelysmall, there appears to be no difference to the naked eye in the image,but with an apparatus according to the invention, the displacement ofthe points may be read and the information in the image may be decoded.As mentioned, the coded information can be, for example, positions thatmake it easier to scan in the image, or text, for example, anexplanation of the image, or copyright information.

To summarize, these two aspects of the invention illustrate how aninformation-coding pattern may be used to create an image with which auser can interact. As will be discussed further below, the informationcontained in the raster of an image will be readable with the help of areading device. An example of an application of the invention accordingto the above discussed aspects, is to produce, e.g., newspapers ormagazines wherein images are printed with a coding pattern as discussed.A reader/user will then, depending on the information contained in thecoding pattern, be able to “interact” with the images to the extent thatthe reading device will be able to convey the further informationcontained in the image.

Such an application is envisaged in a system for handling digitizedinformation, wherein the system comprises a database of informationrelating to a virtual area consisting of all position coordinates that aposition coding pattern described herein is capable of encoding. Thevirtual area is separated into regions, to each of which rules areassociated, where the rules state how the system is to handle encodedinformation emanating from an apparatus reading the coding pattern.Regions associated with different rules may, or may not, be visible tothe eye when printed on a product.

An apparatus for reading the coding pattern hence communicates with,e.g., a server in a computer network or with a more local computer, inwhich server or computer a matching procedure takes place betweenreceived position codes and the database. After the matching, actionsare taken in accordance with the rules associated with the regionmatching the received position coding information. Examples of suchactions include further information retrieval and presentation fromcomputer servers, via e.g. Internet, associated with a printed image in,e.g., a magazine, catalogue or book. The further presentation ofinformation may be of any “multi-media” character and include images,moving images and sound. Other actions, which may be handled accordingto the rules, include interaction between the reader/viewer and an actorassociated with the region in question. Such interaction may be anyelectronic commerce interactions such as ordering products, financialservices etc.

In the context of the invention, the term “image” should be given abroad interpretation, including all kinds of graphic reproductions.Apart from photographic reproduction, images in the form of traditionalforms and layouts may be created, where lines and boxes are visualizedby the image in the form of the coding pattern. Examples of such formsare to be found in applicant's international patent applications WO00/73981, WO 00/73887 and WO 01/16691, all of which are hereby includedby reference.

A specific advantage of the invention is that, by printing an imageusing a coding pattern containing further information which enablesinteraction, it allows for more flexibility when providing products withimages. More space can be appointed to images as compared to traditionalpresentations where barcodes, tick-boxes etc. are used. Moreover, it isadvantageous that the printing of the image and the coding pattern isperformed in one and the same step.

Needless to say, a product according to the invention can be made moreaesthetically pleasing to view in that the image itself comprises theregions allowing the interaction.

A general advantage of pattern and pattern production according to theaspects of the invention is more or less related to the fact that thisis a digital operation. The positions of the markings are entered andinterpreted as binary numbers, in which case the equipment to be usedcan be of relatively simple nature.

According to the aspects of the invention pattern and pattern productionmay be more or less related to the fact that this may be a digitaloperation. The positions of the markings may be entered and interpretedas binary numbers, in which case the equipment to be used can be ofrelatively simple nature.

A more or less problematic situation which can arise in the printing ofmarkings, i.e. screen-printing points, with varying spatial extents maybe of course that the markings can overlap one another in certain areas.These areas may be characterized by a relatively high density. Suchoverlaps may make it more difficult to determine the location of singlemarkings and thus make it more difficult to read the additionalinformation coded in the markings. Certainly, the problem can be avoidedby markings that can be expected to overlap being limited in theirspatial extents even before the printing. However, this entails limitingthe dynamic range with respect to the density variation thatcharacterizes the image printed with the coding pattern.

Known solutions with respect to the determination of the locations ofoverlapping markings can be drawn from, for example, the technique ofastrometrically determining positions of imaged stars that are locatedin dense star fields. This technique tells us that with knowledgegathered in advance about the appearance, in the form of intensitydistribution within the markings, of single markings, the positions ofthe markings, i.e. the positions of the stars, can be determined even ifa plurality of the markings overlap one another. Three-dimensionalpattern recognition algorithms are applied that more or less directlygive, for example, the center point of a marking that “flows together”with adjoining markings.

A disadvantage of astrometrical techniques is that they presupposeadvance knowledge of the three-dimensional appearance of the markingsand their intensity distribution. This assumption may be the only wayfor a pattern recognition algorithm to deliver a reasonable assumptionregarding the center point of the marking.

With an appearance, i.e., spatial extent (form) and size, determined inadvance for each one of the markings that are to constitute a pattern, apattern can be produced according to the invention. Markings may beprovided with changes determined in advance. The changes occuradvantageously in the spatial configuration of the markings and can bemore or less regular. The changes may preferably be so regular that itmay be possible in subsequent reading of the pattern with its markingsto recognize the markings, when an identification of the change is made,without necessarily having read the marking in its complete spatialextent. Once the marking is recognized, for example, its center pointcan be determined.

The changed markings can imply, for example, that overlapping markingscomprise at least one contrasting indicator, the spatial extent andplace within the marking that indicate the center point of the marking.Certainly, these contrasting indicators can be of more or less arbitraryconfiguration but it certainly suggests itself to utilize simple symbolssuch as points, circles or rectangles since reading and interpretingsuch symbols may require a limited set of image analysis operations todetect them.

An advantage of patterns and producing patterns according to the abovemay be that a large dynamic range of density levels can be obtainedwhile retaining the readability. It may be thus possible to printpatterns that contain large markings/raster points without, for example,losing the digitally coded position information.

According to the third aspect of the invention, a coding pattern locatedin the image and containing symbols may be read. As according to theabove aspects, the symbols may represent at least two different values,each symbol comprising one raster point and at least one marking. Theraster point may be included in a raster that may extend over thesurface and the value of each symbol may be indicated by the placementof that marking in relation to a raster point. The method may comprisedetermining markings which at least partially overlap one another andreading for each one of these overlapping markings at least onecontrasting indicator, the spatial extent and place within the markingof which indicate the placement of the marking.

Reading can be advantageously carried out by means of an apparatus thatcan be handheld and communicate wirelessly with, for example, a computerthat receives both image information and information from the codingpattern read.

The foregoing summarizes only a few aspects of the invention and is notintended to be reflective of the full scope of the invention as claimed.Additional features and advantages of the invention are set forth in thefollowing description, may be apparent from the description, or may belearned by practicing the invention. Moreover, both the foregoinggeneral description and the following detailed description are exemplaryand explanatory and are intended to provide further explanation of theinvention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate one embodiment of the inventionand, together with the description, serve to explain the objects,advantages, and principles of the invention. In the drawings:

FIG. 1 schematically shows an embodiment of a product according to thepresent invention which may be provided with a position-coding pattern;

FIGS. 2 a-2 d schematically show how the symbols can be configured in anembodiment of the invention;

FIG. 3 schematically shows an example of 4×4 symbols which may be usedfor coding a position in accordance with the invention;

FIG. 4 schematically shows an apparatus according to the presentinvention which can be used for reading a coding pattern;

FIG. 5 a shows an image in the form of a coding pattern includingmarkings with varying sizes in accordance with the invention;

FIG. 5 b shows a detail from the image of FIG. 5 a;

FIG. 5 c shows the image of FIG. 5 a printed according to prior art; and

FIGS. 6 a and 6 b show details from an image where overlapping markingshave been provided with contrasting indicators in accordance with theinvention.

DESCRIPTION OF PREFERRED EMBODIMENTS

For the sake of clarity, the detailed description of the invention belowmay be divided into a number of part-descriptions. As an introduction, acoding pattern will be presented with reference to FIGS. 1, 2 a-d and 3.This coding pattern can be used for information storage as has beenoutlined above. As an example, the information stored in the pattern mayinclude position information. After the presentation of the codingpattern, an apparatus intended for reading the pattern is then presentedin connection with FIG. 4. After that is shown how an image can beproduced by printing a screen pattern in the form of the coding patternpresented, comprising markings of varying extents, referring to FIGS. 5a and 5 b. Finally, a method of modifying the appearance of the patternmarkings that, on printing an image, overlap one another, is presentedwith reference to FIGS. 6 a and 6 b.

In the following discussion regarding the coding pattern a number ofcalculations take place. These calculations are performed in aprocessing means, such as a processor connected to a printing device.The software that performs the calculations is written in accordancewith the description to follow and it is assumed that the person skilledin the art will choose suitable tools for performing such programming.The software may be present, or stored, in any form known in the art,such as any volatile or non-volatile memory units capable of beingconnected to the processor and be readable by the processor, such as adiskette or a CD-ROM, as well as propagated signals such as the streamof bits that represent Internet transmission of packets or the carrierwaves that are transmitted to satellites.

The coding pattern used below is described in some more detail in ourown disclosures PCT/SE00/01895, PCT/SE00/01897 and WO 01/16691 herebyalso incorporated by reference.

FIG. 1 shows a part of a product in the form of a sheet of paper 1.Sheet 1 may be may include a principle surface 2 with an opticallyreadable position-coding pattern 3. An apparatus can readposition-coding pattern 3 to perform position determination on thesurface. The position-coding pattern may further include symbols 4 thatmay be systematically arranged over the surface 2, giving it a“patterned” appearance. The symbols may include markings that, as shownin FIG. 1, may be round and of constant size. This depiction isexemplary and not exhaustive of the type of shapes that may be employedin accordance with the invention.

Screen-printed images may be made up of a large number of pictureelements in the form of points i.e., raster points. An image may becharacterized by spatially varying density, with variations in densityrepresenting by varying extents of the raster points. Depending on theequipment used for generating the images with the aid of this technique,a widely varying quality in terms of spatial resolution and in terms ofnumber of levels of density may be produced. A determining factor may bethe capacity of the equipment with respect to the size of the rasterpoints and the nature of the surface on that the image may be printed.The surface could be a flat sheet of paper, a rounded container, etc.

The effect that may be achieved according to the exemplary embodiment isthat additional information may be obtained from an image printed on aproduct with a screen-printing technique. Recognizing the relationshipallows one to develop a coding pattern that permits image to hold alarge amount of information. Images without an information-carryingcoding pattern would lack the additional “layer” of information that maybe contained in the coding pattern.

The paper 1 has an x-coordinate axis and a y-coordinate axis. In thiscase, the position determination can be performed on the surface of theentire product. In other cases, the surface may constitute a smallerportion or section of the product.

The position-coding pattern 3 may include a virtual raster that may beneither visible to the human eye nor detectable directly by an apparatusthat is to determine positions on the surface. Further, as explainedbelow, a plurality of symbols 4 within the pattern, may assume one offour values. In this example, the values are “1”-“4”. In thisconnection, it should be pointed out that the position-coding pattern inFIG. 1 is greatly enlarged for the sake of clarity. Moreover, it isshown only on part of the paper.

Position-coding pattern 3 may be arranged such that the position of apartial surface on the principle surface 2 is coded by symbols. A firstand a second partial surface 5 a, 5 b are shown with dashed lines inFIG. 1. The part of the position-coding pattern 30 (here 3×3 symbols)located on the first partial surface 5 a coding or representing a firstposition and the part of the position-coding pattern which may belocated on the second partial surface 5 b codes a second position. Theposition-coding pattern 3 may be thus partly common to the adjoiningfirst and second positions. Such a position-coding pattern may bedesignated as “floating” in this application.

FIGS. 2 a-d show an embodiment of a symbol that can be used in theposition-coding pattern according to the invention. The symbol mayinclude a virtual raster point 6, represented by the intersectionbetween the raster lines, and a marking 7. The value of the symbol maybe based on where the marking is placed. In the example in FIG. 2, fourpossible placements are located, one on each of the raster lines thatstart from the raster points 6. The displacement of marking 7, or morespecifically its center of gravity, from the raster point 6 may be ofthe same magnitude for all values. In the text that follows, the symbolhas the value 1 in FIG. 2 a, the value 2 in FIG. 2 b, the value 3 inFIG. 2 c and the value 4 in FIG. 2 d. Expressed differently, there arefour different types of symbols.

Each symbol can also represent four values “1-4”. This entails that theposition-coding pattern can be divided into a first position code forthe x-coordinate and a second position code for the y-coordinate. Thedividing may be carried out according to the following: Symbol value xcode y code 1 1 1 2 0 1 3 1 0 4 0 0

The value of each symbol may thus be translated into a first digit, inthis case bit, for the x code and into a second digit, in this case bit,for the y code. In this way two completely independent bit patterns areobtained. The patterns can be combined into a common pattern that may begraphically coded by means of a plurality of symbols according to FIG.2.

Each position may be coded by a plurality of symbols. In this example,4×4 symbols are used for coding a position in two dimensions, i.e. anx-coordinate and a y-coordinate.

The position code may be made up by means of a number series of ones andzeros that have the characteristic that no sequence of four bits occursmore than once in the series. The number series may be cyclic whichmeans that the characteristic also applies if the end of the series maybe coupled together with its beginning. A sequence of four bits thusalways has an unambiguously determined position in the number series.

The series can be maximally 16 bits long if it has the characteristic ofsequences of four bits described above. In this example, however, only aseven-bits-long series according to the following may be used:

-   -   “0 0 0 1 0 1 0”

This series contains seven unique sequences of four bits which code aposition in the series according to the following: Position in theseries Sequence 0 0001 1 0010 2 0101 3 1010 4 0100 5 1000 6 0000

For coding the x-coordinate, the number series may be writtensequentially in columns over the entire surface to be coded. The codingmay be based on the difference or position displacement between numbersin adjoining columns. The magnitude of the difference may be determinedby the position in the number series at which the column is to begin(i.e. with which sequence). More specifically, if the difference moduloseven may be taken between, on the one hand, a number coded by afour-bit sequence in a first column and which can thus have the value(the position) 0−6, and, on the other hand, a corresponding number (i.e.the sequence of the same “height”) in an adjoining column, the resultwill be the same independently of where along the two columns thecomparison may be made. Thus, it may be possible to code an x-coordinatethat may be constant for all y-coordinates by means of the differencebetween two columns.

Since each position on the surface may be coded by 4×4 symbols in thisexample, this may provide access to three differences (with the value0−6), according to the above for coding the x-coordinate. The coding maythen be done in such a manner that of the three differences, one willalways have the value 1 or 2 and the other two will have the value inthe interval 3−6. No differences will thus be zero in the x code. Inother words, the x code may be constructed in such a manner that thedifferences will be as follows:

(3−6) (3−6) (1−2) (3−6) (3−6) (1−2) (3−6) (3−6) (1−2) . . .

Each x-coordinate may be thus coded with two numbers between 3 and 6 anda subsequent number that may be 1 or 2. Subtracting 3 from the highnumbers and one from the low one, a number may be obtained in a mixedbase that directly provides a position in the x direction from which thex-coordinate can then be determined directly as is shown in the examplebelow.

Using the principle described above, it may be thus possible to codex-coordinates 0, 1, 2 . . . , by means of numbers that represent threedifferences. These differences may be coded with a bit pattern that maybe based on the above number series. Finally, the bit pattern can becoded graphically by means of the symbols in FIG. 2.

In many cases, when inputting 4×4 symbols, one does not obtain acomplete number that codes the x-coordinate but instead parts of twonumbers. Since the least significant part of the numbers may be obtainedfrom these values, however, a complete number can be reconstructed in asimple manner.

The y-coordinates may be coded in accordance with the same principleused for the x-coordinates. The cyclic number series may be writtenrepeatedly in horizontal rows over the surface to be position-coded.Exactly as in the case of the x-coordinates, the rows are allowed tobegin at different positions, i.e. with different sequences, in thenumber series. For the y-coordinates, however, differences may not beused but the coordinates may be coded with numbers that are based on thestarting position of the number series in each row. Having determinedthe x-coordinate for 4×4 symbols, it may be possible to determine thestarting positions in the number series for the rows that may be the ycode in the 4×4 symbols. In the y code, the most significant number maybe determined by allowing it to be the only one that has a value in aspecific interval. In this example, one row of four is allowed to beginin position 0−1 in the number series to indicate that this row relatesto the least significant number in a y-coordinate, and the other threebegin in position 2−6. In the y direction, there may be a number seriesaccording to the following:

(2−6) (2−6) (2−6) (0−1) (2−6) (2−6) (2−6) (0−1) (2−6) . . .

Each y-coordinate may be coded with three numbers between 2 and 6 and asubsequent number between 0 and 1.

Subtracting 1 from the low number and 2 from the high ones provides inthe same manner as for the x direction a position in the y direction inmixed base from which the y-coordinate may be determined directly.

Using the above method, 4×4×2=32 positions can be coded in the xdirection. Each such position may correspond to three differences thatprovide 3×32=96 positions. Furthermore, 5×5×5×2=250 positions can becoded in the y direction. Each such position may correspond to 4 rowsthat provide 4×250=1000 positions. Altogether, 96,000 positions can thusbe coded. Since the x coding may be based on differences it may bepossible to select the position where the first number series begins.Taking into consideration that the first number series can begin atseven different positions, it may be possible to code 7×96,000=672,000positions. The starting positions for the first number series in thefirst column can be calculated when the x-coordinate has beendetermined. The above-mentioned seven different starting positions forthe first series can code different sheets or writing surfaces on aproduct.

For further illustrating the invention according to this embodiment, aspecific example follows here that may be based on the embodiment of theposition coding described.

FIG. 3 shows an example of an image with 4×4 symbols that may be read byan apparatus for position determination in accordance with the presentinvention.

These 4×4 symbols have the following value:

4 4 4 2

3 2 3 4

4 4 2 4

1 3 2 4

These values represent the following binary x and y code: x code: ycode: 0 0 0 0 0 0 0 1 1 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 1 1 0 0 1 0 1 0

The vertical x sequences code the following positions in the numberseries: 2 0 4 6. The differences between the columns become −2 4 2 whichmodulo 7 provides: 5 4 2 which, in mixed base, codes position(5−3)×8+(4−3)×2+(2−1)=16+2+1=19. Since the first coded x position may beposition 0, the difference that lies in the interval 1−2 and thatappears in the 4×4 symbols may be the twentieth such difference. Sincethere may also be a total of three columns for each such difference andthere may be a start column, the vertical sequence furthest to the rightin the 4×4 x code belongs to the 61st column in the x code (3×20+1=61)and that furthest to the left to the 58th.

The horizontal y sequences code the positions 0 4 1 3 in the numberseries. Since these series begin in the 58th column, the startingposition of the rows may be these numbers minus 57 modulo 7 that mayprovide the starting positions 6 3 0 2. Translated into digits in themixed base, this becomes 6−2, 3−2, 0−0, 2−2=4 1 0 0, where the thirddigit may be the least significant digit in the current number. Thefourth digit may then be the most significant digit in the next number.In this case, this must be the same as in the current number. Anexception being when the current number consists of the highest possibledigits in all positions. It is then apparent that the start of the nextnumber may be one greater than the start of the current number.

The position of the four-digit number becomes 0×50+4×10+1×2+0×1=42 inthe mixed base.

The third row in the y code may be the 43rd that may have a startingposition 0 or 1 and since there may be four rows in total for each suchrow, the third row is number 43×4=172. In this example, the position forthe top left corner of the 4×4 symbol group may be 58,170.

Since the x sequences in the 4×4 group begin in row 170, the x columnsof the entire pattern begin in positions ((2 0 4 6)-169) mod 7=1 6 3 5of the number series. Between the last starting position (5) and thefirst starting position, the numbers 0-19 may be coded in the mixed baseand by summing up the representations for the numbers 0-19 in the mixedbase, the total difference between these columns may be obtained. Aprimitive algorithm for doing this calculation may generate these twentynumbers and directly sum up their digits. If the sum obtained is calledS, the sheet or writing surface may then be given by (5-S) modulo 7.

In the above example, an embodiment has been described where eachposition may be coded with 4×4 symbols and a number series with 7 bitsmay be used. Naturally, this is only one example. Positions can be codedwith more or fewer symbols. They do not need to be the same amount inboth directions. The number series can have different lengths and doesnot need to be binary but can be based on another base. Different numberseries can be used for coding in the x direction and coding in the ydirection. The symbols can have a different number of values.

Furthermore, in the above example the marking is a point. Naturally, itcan have other appearances. For example, it can be in the form of a linethat begins at the virtual raster point and extends from there to acertain position.

In the above example, the symbols are used within a square partialsurface for coding a position. The partial surface can have a differentshape, for example, hexagonal. Neither do the symbols need to bearranged in rows and columns at 90° angles with respect to one anotherbut can also be arranged in other configurations.

Depending on the details of the embodiment, for the position code to bedetectable, the virtual raster may need to be determined. This may bedone by studying the distance between different markings. The shortestdistance that is found between two markings must originate from twoadjoining symbols with the value 1 and 3 so that the markings are lyingon the same raster line between two raster points. Once such a pair ofmarkings has been detected, the associated raster points can bedetermined with knowledge of the distance between the raster points andthe displacement of the markings from the raster points. When two rasterpoints have been localized, additional raster points can be determinedmeasuring distances to other markings and with knowledge of the mutualdistances between the raster points.

FIG. 4 schematically shows an embodiment of an apparatus for positiondetermination. It may include a casing 11 that may have the approximateform of a pen, or similar writhing instrument. In the short end of thecasing, an opening 12 may be located. The short end may be intended tobear against, or be held at a small distance from, the surface to whichan image may be applied and the position determination or more generalinformation gathering is to take place.

The casing may include an optics part, an electronic circuitry part anda power supply. The optics part may include at least one light-emittingdiode 13 for illuminating the surface that may be imaged and alight-sensitive area sensor 14, for example a CCD or CMOS sensor, forregistering a two-dimensional image. The apparatus may also contain alens system. The power supply for the apparatus may be obtained from abattery 15 that may be mounted in a separate compartment in the casing.

The electronic circuitry part may include an image-processing element 16for determining a position on the basis of the image recorded by thesensor 14 and, more specifically, a processor unit with a processor thatmay be programmed for reading images from the sensor and carrying outthe position determination on the basis of these images.

Thus, the apparatus in this embodiment can also comprise a pen point 17with the aid of which one can write normal pigment-based writing on thesurface on which the position determination is to take place. The penpoint 17 may be retracted and extended so that the user can control itsuse. In certain applications, the apparatus does not need to have anypen point at all or may employ other types of writing implements thatare not necessarily pigment-based. In certain applications, theapparatus does not need to have any pen point at all or comprise afixed, non-retractable, pen point. Moreover, the device may comprise aplurality of retractable pen points, each having, e.g., a differentcolor.

The apparatus also may include buttons 18 to activate and control theapparatus. It may also have a transceiver 19 for wireless transmission,e.g., by IR light or radio waves, of information to and from theapparatus. The apparatus can also include a display 20 for showingpositions or recorded information.

In Swedish Patent No. 9604008-4 (herein incorporated by reference) ofthe Applicant, an apparatus for recording text is described. Thisapparatus can be used for position determination if it is programmed ina suitable manner. If it is to be used for pigment-based writing, it canalso be provided with a pen point, or similar writing instrument.

The apparatus can be divided into different physical casings, a firstcasing containing components necessary for capturing images of theposition-coding pattern and for transferring these to components locatedin a second casing and perform the position determination on the basisof a recorded image or images.

As mentioned, the position determination is done by a processor havingsoftware for locating and decoding the symbols in an image and fordetermining positions from the codes thus obtained. From the aboveexample, a person skilled in the art can design software that carriesout position determination on the basis of an image of a part of aposition-coding pattern. Furthermore, a person skilled in the art candesign software for printing the position-coding pattern on the basis ofthe above description.

In the embodiment above, the pattern can be read optically and thesensor may be optical. As mentioned, the pattern can be based on anotherparameter than an optical parameter. In this case, the sensor should beof a type that can read the current parameter. Examples of suchparameters are of chemical, acoustical or electromagnetic character,e.g. resistance, capacitance and inductance. Further, in the embodimentabove, the raster is a rectangular grid. It can also have other forms,i.e. non-rectangular.

In the embodiment above, the raster is a grid network, but it can alsohave other forms. In the embodiment above, it is not the longestpossible cyclic number series that is used. This results in a certainredundancy that can be used, for example, for checking the turning ofthe read group of symbols.

The actual calculations relating to the reading of the coding patternare performed in processing means, such as those discussed above inconnections with FIG. 4, may be a processor within the device itself. Itis also feasible to perform the calculations in an external processorconnected to the reading device. The software that performs thecalculations may be written in accordance with the present disclosure tofollow and it is assumed that the person skilled in the art will choosesuitable tools for performing such programming. The software may bepresent, or stored, in any form known in the art, such as any volatileor non-volatile memory units capable of being connected to theprocessor, such as a diskette or a CD-ROM, as well as propagated signalssuch as a stream of bits that represent Internet transmission of packetsor carrier waves that are transmitted to satellites.

FIG. 5 a shows an image that may be produced of a coding pattern, wherethe coding pattern may include markings of varying sizes. The symbolsmay be part of a coding pattern according to the description above inconnection with FIGS. 1-3. The aim of FIG. 5 a is to illustrate thatvarying density can be produced with the aid of symbol markingsaccording to the invention. For the sake of clarity, however, the imageis shown at a very coarse scale. Certainly, the sizes of the markingsand thus the detail resolution may depend on the capacity of theprinting device that generates the image printout as can be easilyunderstood by a person skilled in the art.

FIG. 5 b shows a detail from the image in FIG. 5 a and shows moreclearly than in FIG. 5 a how the markings can vary in spatial extents.This spatial extent may be viewed as the shape, construction, densityand/or relative position to other markings. A large number of markingshave such positions, determined by the coding, and extents so that theyoverlap adjacent markings. FIG. 5 b provides an example of an area 501that contains such overlappings. It should be noted that the markings asshown in FIGS. 5 a and 5 b do not have the same general appearance asthe markings shown in FIGS. 1-3 that are more or less circular. Thisillustrates the fact that, within the scope of this invention, theappearance of the markings is not limited to some specific shape but canadvantageously depend on the capacity of the printing device. Theexamples in FIGS. 5 a and 5 b originate from a conventional laserprinter with limited detail resolution that is why the shapes of themarkings vary.

In FIG. 5 c, the image of FIG. 5 a is printed according to conventionaltechnique, with no coding pattern involved. As can be seen, by comparingwith the image of FIG. 5 a, introduction of the coding pattern leads tomerely a slight degradation of the quality of the image itself.

FIGS. 6 a and 6 b show schematic illustrations of markings 61, 62, 65,66, 67 that overlap one another. The markings are presented in the formof filled-in circles. For the sake of clarity, the fillings have beencarried out by means of shading in contrast to the solidly filled-inmarkings illustrated earlier in FIGS. 1-3 and 5. The markings in FIGS. 6a and 6 b have also been provided with contrasting indicators 63, 64,68, 69, and 70.

The overlapping markings 61, 62 in FIG. 6 a may be provided withrespective circular contrasting indicators 63, 64, placed in the centerof the respective marking. The overlapping markings, shown as 65, 66,67, may be further provided with respective contrasting indicators 68,69, 70 in the form of circles that may be concentric with the respectivemarking and centered with respect to the center of the respectivemarking.

On generating an image such as a picture, where, due to the density oneor more markings spatially overlap, the markings may be provided with acontrasting indicator. The generation of markings is preferably done bymeans of software in a suitably programmed computer. The generatingprocess may include determining, with knowledge of the density of eachpicture element, the extents of the markings and deciding if adjacentmarkings are to overlap one another on printing or printout. If suchoverlappings are found, these markings may be provided with acontrasting indicator according to, for example, one of theillustrations in FIG. 6 a or 6 b. However, it should be pointed out onceagain that the markings and the contrasting indicators could be of moreor less arbitrary form.

Inputting of symbols having markings with contrasting indicators maypreferably be done with a computer provided with suitably designedsoftware. An inputting process may include detecting two or moreoverlapping markings. The detection can include, for example, inputtingof a spot of undetermined spatial configuration, the appearance ofwhich, for example, can be one of the illustrations in FIG. 6 a or 6 b.On such a finding, the software may analyze the appearance of the spotwith a known image-analysis technique, to identify contrastingindicators. These contrasting indicators, for example, may then beanalyzed in terms of their position and extent, a center point of therespective indicators being found and interpreted as a center point of arespective marking.

Although the storage of position information in a coding pattern, andreading of position information from a coding pattern, is shown in theabove example, other information can also be relevant. Text in the formof an explanation of the image or copyright information may also providerelevant information.

Concurrently filed with the application for this patent are applicationsentitled Systems and Methods for Information Storage based on SwedishApplication No. 0000947-2, filed Mar. 21, 2000, and U.S. ProvisionalApplication No. 60/207,839, filed May 30, 2000; Secured Access Using aCoordinate System based on Swedish Application No. 0000942-3, filed Mar.21, 2000, and U.S. Provisional Application No. 60/207,850 filed on May30, 2000; System and Method for Printing by Using a Position CodingPattern based on Swedish Application No. 0001245-0, filed on Apr. 5,2000, and U.S. Provisional Application No. 60/210,651, filed on Jun. 9,2000; Apparatus and Methods Relating to Image Coding based on SwedishApplication No. 0000950-6, filed on Mar. 21, 2000, and U.S. ProvisionalApplication No. 60/207,838, filed on May 30, 2000; Apparatus and Methodsfor Determining Spatial Orientation based on Swedish Application No.0000951-4, filed on Mar. 21, 2000, and U.S. Provisional Application No.60/207,844, filed on May 30, 2000; System and Method for DeterminingPositional Information based on Swedish Application No. 0000949-8, filedMar. 21, 2000, and U.S. Provisional Application No. 60/207,885, filed onMay 30, 2000; Method and System for Transferring and DisplayingGraphical Objects based on Swedish Application No. 0000941-5, filed Mar.21, 2000, and U.S. Provisional Application No. 60/208,165, filed May 31,2000; Online Graphical Message Service based on Swedish Application No.0000944-9, filed Mar. 21, 2000, and U.S. Provisional Application No.60/207,881, filed May 30, 2000; Method and System for DigitizingFreehand Graphics With User-Selected Properties based on SwedishApplication No. 0000945-6, filed Mar. 21, 2000, U.S. ProvisionalApplication No. 60/207,882, filed May 30, 2000; Data Form Having aPosition-Coding Pattern Detectable by an Optical Sensor based on SwedishApplication No. 0001236-9, filed Apr. 5, 2000, and U.S. ProvisionalApplication No. 60/208,167, filed May 31, 2000; Method and Apparatus forManaging Valuable Documents based on Swedish Application No. 0001252-6,filed Apr. 5, 2000, and U.S. Provisional Application No. 60/210,653filed Jun. 9, 2000; Method and Apparatus for Information Managementbased on Swedish Application No. 0001253-4 filed Apr. 5, 2000, and U.S.Provisional Application No. 60/210,652, filed Jun. 9, 2000; Device andMethod for Communication based on Swedish Application No. 0000940-7,filed Mar. 21, 2000, and U.S. Provisional Application No. 60/208,166,filed May 31, 2000; Information-Related Devices and Methods based onSwedish Application No. 0001235-1, filed Apr. 5, 2000, and U.S.Provisional Application No. 60/210,647, filed Jun. 9, 2000; Processingof Documents based on Swedish Application No. 0000954-8, filed Mar. 21,2000, and U.S. Provisional Application No. 60/207,849, filed May 30,2000; Secure Signature Checking System based on Swedish Application No.0000943-1, filed Mar. 21, 2000, and U.S. Provisional Application No.60/207,880, filed May 30, 2000; Identification of Virtual RasterPattern, based on Swedish Application No. 0001235-1, filed Apr. 5, 2000,and U.S. Provisional Application No. 60/210,647, filed Jun. 9, 2000, andSwedish Application No. 0004132-7, filed Nov. 10, 2000, and U.S.Provisional Application No. ______, filed Jan. 12, 2001; and a new U.S.Provisional Application entitled Communications Services Methods andSystems.

The technical disclosures of each of the above-listed U.S. applications,U.S. provisional applications, and Swedish applications are herebyincorporated herein by reference. As used herein, the incorporation of a“technical disclosure” excludes incorporation of informationcharacterizing the related art, or characterizing advantages or objectsof this invention over the related art.

The foregoing description is presented for purposes of illustration anddescription. It is not exhaustive and does not limit the invention tothe precise form disclosed. Modifications and variations are possible inlight of the above teachings or may be acquired from practicing theinvention. The scope of the invention is defined by the claims and theirequivalents.

1. A system for handling digitized information, comprising: aninteractive image which comprises human-readable information and amachine-readable code embedded in one and the same layer of printing inkon a product surface; a reading device configured to detect at leastpart of the machine-readable code in the interactive image; and acomputing device configured to identify an action to be taken as afunction of the detected machine-readable code.
 2. The system of claim1, wherein said action comprises at least one of retrieving furtherinformation, presenting further information to a user of the readingdevice, ordering products, and ordering financial services.
 3. Thesystem of claim 2, wherein said further information comprises multimediainformation.
 4. The system of claim 1, wherein the computing device isconfigured to identify said action by matching at least one positionderived from the machine-readable code against a database in whichpositions are associated with information handling rules.
 5. The systemof claim 1, wherein the machine-readable code comprises aposition-coding pattern.
 6. The system of claim 5, wherein theposition-coding pattern comprises a plurality of partial surfaces, eachencoding a unique position in a predetermined coordinate system.
 7. Thesystem of claim 6, wherein the reading device, when placed by a userinto an operative position to the interactive image, is configured todetect one of said partial surfaces.
 8. The system of claim 6, whereinsaid partial surfaces are partly overlapping for mutually adjacentpositions in said coordinate system.
 9. The system of claim 6, whereineach partial surface comprises a plurality of code symbols.
 10. Thesystem of claim 9, wherein each code symbol comprises a raster point andat least one marking; the raster point being included in a raster whichextends across the interactive image; at least one value of each codesymbol being indicated by a displacement of a center point of the atleast one marking in relation to a raster point.
 11. The system of claim9, wherein the value of each code symbol is translatable into at leastone first number which is used for coding a first position coordinateand at least one second number which is used for coding a secondposition coordinate, the code symbols in the machine-readable codetogether representing a first position code for the first positioncoordinate and a second position code for the second positioncoordinate.
 12. The system of claim 1, wherein the machine-readable codeincludes code symbols representing at least two different values, eachcode symbol comprising a raster point and at least one marking; theraster point being included in a raster which extends over the productsurface; at least one value of each code symbol being indicated by adisplacement of a center point of the at least one marking in relationto a raster point; and the markings each having a spatial extent which,in combination, at least partially form the human-readable information.13. The system, of claim 12, wherein the markings have essentiallyidentical shape and varying size.
 14. The system of claim 12, wherein atleast markings that partially overlap one another comprise at least onecontrasting indicator that indicates a center point of the marking. 15.The system of claim 1, wherein the machine-readable code is based on afirst cyclic number series which has the characteristic that no sequencewith a first predetermined length occurs more than once in the numberseries.
 16. The system of claim 1, wherein the machine-readable codeencodes a first position coordinate by a first cyclic number seriesbeing repeated in columns in the interactive image, the columnsbeginning at different places in the number series, and the first cyclicnumber series having the characteristic that no sequence with a firstpredetermined length occurs more than once in the number series.
 17. Thesystem of claim 1, wherein the machine-readable code encodes a secondposition coordinate by a second cyclic number series being repeated inrows in the interactive image, the rows beginning at different places inthe number series, and the second cyclic number series having thecharacteristic that no sequence with a second predetermined lengthoccurs more than once in the number series.
 18. The system of claim 1,wherein the machine-readable code encodes at least one of a position, anumber, and a text related to the human-readable information.
 19. Thesystem of claim 1, wherein the human-readable information comprises aphotographic reproduction.
 20. The system of claim 1, wherein thereading device is handheld.
 21. A product which has a surface providedwith an interactive image which comprises human-readable information anda position-coding pattern embedded in one and the same layer of printingink on the surface, wherein the position-coding pattern comprises aplurality of partial surfaces, each encoding a unique position in apredetermined coordinate system.
 22. A method of enabling a user toinstruct a computing device via: a printed image which compriseshuman-readable information and a machine-readable code embedded in oneand the same layer of printing ink on a product surface; and a readingdevice which detects at least part of the machine-readable code andgenerates coding information; the method comprising the steps of, in thecomputing device: receiving the coding information from the readingdevice; identifying an action to be taken as a function of the codinginformation; and initiating the action.